Amortized SVGD (Stein Variational Gradient Descent)
Stein variational gradient descent (SVGD) [1] is a deterministic, gradient-based sampling algorithm for approximate inference. Given a probability density function by a simple iterative update of form.
Compared with Monte Carlo methods, SVGD can achieve good approximation even with a very small number of particles. A simple way to see this is to note that when using only a single particle, which is often found to be a useful approximation in many difficult practical problems.
SVGD with more particles interpolates between gradient descent and approximate inference and provides better uncertainty assessment.